JTji 


ANNUAL  TECHNICAL  WONT  WIHtOHH 


MENTAL  REPRESENTATION  OF  ORCUIT  DIAGRAMS: 
MDIVDUAL  DEFERENCES  M  STRUCTURAL  KNOWLEDGE 


Thom*  (X 
Rat*  E.  QiMm 

MtahMia  SNMk 

I  VAkAyNM 

jflMMI  TMWIDRR 


OFFICE  OF  NAVAL  RE8EAR 
800  North  Quincy  Erirt 
Arihglan,  VA  22217 


PERCEPTRONICS 


*271  VARICL  AVSNUI  •  WOODLAND  HILL*  •  CALIFORNIA  *1**7  •  RNONS  <«*)  **4.747* 


REPORT  DOCUMENTATION  PACE 


«.  nn.* 


Mental  Representation  of  Circuit  Diagrams: 
Individual  Differences  In  Structural  Knowledge 


Annual  Report 


PAIR- 11 09-82-6 


THoms  D.  Wlckens,  Ralph  E.  Gelselman, 
Michael  6.  Sonet,  A  Cynthia  L.  Yelvlngton 


Perceptronlcs,  Inc. 

6271  Varlel  Avenue 
Woodland  Hills,  CA  91367 


11.  COnTMOIUN*  OWICS  NAM  ANO  AM 

Office  of  Naval  Research 
800  North  Quincy  Street 
Arlington,  VA  22217 


N0001 4-81 -C-0S90 


1*.  RCNONT  OAT* 

June  1982 


tl.  KEY  WORM  (OmNAm  m  <inw> 

circuit  diagrams 
clustering  algorithms 
cognitive  models 
electronic  circuits 
expert  systems 


f<— Wfr  If 

circuit  diagrams  functional  knowledge  structural  knowledge 

clustering  algorithms  Individual  differences  symbolic  knowledge 

cognitive  models  knowledge  representation 

electronic  circuits  mental  representation 

expert  systems  ordered  trees  _  _ 


This  work  Is  concerned  with  the  knowledge  that  electronics  technicians  possess 
of  electronic  equipment,  and  more  generally,  with  how  people  operate  In  tasks 
that  draw  upon  a  complex  spatial  symbolic  knowledge  base.  A  technician's  know¬ 
ledge  base  Is  postulated  to  consist  of  three  types  of  related  knowledge: 

(a)  structural/functional  knowledge,  which  pertains  to  the  actual  configuration 
of  a  circuit  and  the  role  that  Its  components  play  In  the  operation  of  the 
device;  (b)  prototypical  knowledge,  which  pertains  to  the  general  properties 
common  to  circuits  of  a  given  type;  and  (c)  procedural  knowledge,  which  pertain 
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ABSTRACT 


This  work  Is  concerned  with  the  knowledge  that  electronics  technicians 
possess  of  electronic  equipment,  and  more  generally,  with  how  people 
operate  In  tasks  that  draw  upon  a  complex  spatial  symbolic  knowledge 
base.  A  technician's  knowledge  base  Is  postulated  to  consist  of  three 
types  of  related  knowledge:  (a)  structural /functional  knowledge,  which 
pertains  to  the  actual  configuration  of  a  circuit  and  the  role  that  Its 
components  play  In  the  operation  of  the  device;  (b)  prototypical 
knowledge,  which  pertains  to  the  general  properties  common  to  circuits 
of  a  given  type;  r*d  (c)  procedural  knowledge,  which  pertains  to  the  way 
that  a  circuit  can  be  modified  and  to  the  Interaction  among  knowledge 
elements  of  all  three  types  of  knowledge.  The  first  of  these  knowledge 
types,  structural /functional  knowledge,  was  studied  In  an  experiment  In 
which  subjects  were  asked  to  reconstruct  circuits  and  to  partition  cir¬ 
cuits  Into  components.  Dramatic  performance  differences.  In  terms  of 
errors  and  speed,  were  found  between  technicians  of  varying  skill  lev¬ 
els.  However,  any  between-group  differences  In  the  derived  representa¬ 
tions  of  the  structural /functional  knowledge  base  were  dwarfed  by  Indi¬ 
vidual  differences  In  the  patterns  of  reconstruction  and  partitioning 
performance.  Therefore,  It  Is  anticipated  that  more  fundamental  differ¬ 
ences  among  technicians  of  varying  skill  levels  lie  In  their  respective 
procedural  knowledge.  The  results  of  this  research  program  should  help 
In  providing  guidelines  for  training  electronic  technicians  to  better 
understand  and  troubleshoot  complex  equipment. 
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INTRODUCTION 


The  performance  of  complex  tasks  such  as  maintaining  mechanical  equip- 
ment,  modifying  and  adapting  computer  programs,  and  troubleshooting  so¬ 
phisticated  electronic  devices  requires  highly  skilled  personnel. 
Furthermore,  all  such  tasks  require  that  these  personnel  have  a  detailed 
understanding  of  the  devices  that  are  being  repaired  and  maintained. 
Thus,  knowledge  about  how  people  mentally  represent  complex  devices  Is 
necessary  for  developing  suitable  personnel  training  programs  and  for 
designing  optimal  man-machine  Interfaces  for  maintenance  equipment. 


The  training  of  personnel  to  troubleshoot  electronic  circuits  has  typi¬ 
cally  been  less  than  successful.  In  a  review  of  the  status  of  troub¬ 
leshooting  in  the  military  services.  Bond  and  Towne  (1979)  state: 

The  main  conclusion  of  this  report... (Is  that)  trouble¬ 
shooting  of  very  complex  systems  Is  difficult  for 
numerous  reasons,  but  the  critical  factor  Is  that  the 
technician's  cognitive  map  of  essential  physical 
relations  (electronic,  hydraulic,  electro-mechnlcal . 
and  so  on)  In  complex  equipment  Is  often  Incomplete, 
vague,  or  Incorrect.  As  long  as  this  Is  so,  any  series 
of  checks  and  test  readings,  though  apparently  well 
motivated  and  accomplished,  cannot  'close  In'  logically 
on  a  faulty  unit.  (pp.  5-6) 


The  premise  underlying  the  present  research  effort  Is  that  training  pro¬ 
grams  may  be  deficient  when  they  fall  to  provide  troubleshooters  with 
the  knowledge  required  to  develop  a  sufficiently  rich  conceptual  struc¬ 
ture  of  the  equipment  with  which  they  are  working.  Accordingly,  the 
research  presented  here  Is  directed  toward  understanding  the  nature  of 
the  knowledge  base  that  Is  necessary  for  the  repair  and  maintenance  of 
complex  electronic  devices. 
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Nature  of  Mental  Representations 

Troubleshooting  entails  the  Isolation  and  repair  of  malfunctioning  com¬ 
ponents  In  a  device  and,  as  such.  Is  a  form  of  problem  solving.  It  may 
be  analyzed  In  terms  of  problem  solving  theories  (see,  e.g.,  Greeno, 
1978,  for  a  review).  Presumably,  the  problem-solving  procedures  operate 
on  a  substrate  of  knowledge,  which  Includes  a  mental  representation  of  a 
device.  This  underlying  structure  must  be  established  before  problem¬ 
solving  theories  can  be  applied. 

It  Is  apparent  that  an  expert's  mental  representation  of  a  complex  dev¬ 
ice  Is  not  Isomorphic  to  a  schematic  drawing  of  the  device.  In  fact, 
mental  representations  contain  much  more  information  than  the  schematic 
Itself.  (See  Brown,  Collins,  &  Harris,  1978,  for  a  similar  point  of 
view.)  For  example,  an  expert  readily  can  identify  functional  units 
within  a  circuit  that  are  not  directly  present  In  the  diagram.  There¬ 
fore,  a  prudent  Initial  stage  of  research  Is  to  model  the  latent 
knowledge  and  the  cognitive  mechnlsms  that  allow  the  expert  to  develop 
an  enriched  mental  representation  from  a  schematic  drawing. 

The  mental  representation  Is  composed  of  a  number  of  Interrelated  and 
overlapping  structures.  These  structures  may  be  hierarchical  In  form, 
although  this  Is  not  crucial.  The  hierarchical  character  arises  from 
the  tendency  to  view  several  components  of  an  electronic  device  as  a 
unit,  without  examining  their  fine  structure  unless  necessary.  For  ex¬ 
ample,  one  may  think  of  a  logic-circuit  component  as  a  flip-flop, 
without  analyzing  It  Into  basic  components  unless  forced  to  do  so.  This 
hierarchical  tendency  has  been  supported  In  modern  Integrated  circuits 
by  the  physical  modularization  of  rather  complex  functions  In  single 
chips  or  modules;  In  computers,  for  example,  CPUs,  Interface  units,  and 
various  hardware  drivers  are  packaged  as  single  units. 
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The  overlapping  nature  of  the  representation  results  from  the  fact  that 
a  particular  component  participates  In  several  organizations  at  once. 

In  some  computer  designs,  for  example,  a  circuit  representing  the  fourth 
bit  of  an  accumulator  may  logically  be  analyzed  as  part  of  that  accumu¬ 
lator.  or  may  be  considered  as  part  of  the  array  of  fourth  bits  over  a 
series  of  registers  such  as  the  accumulator,  program  counter,  etc.  (It 
Is  Interesting  that  both  forms  of  organization  are  reflected  In  the  phy¬ 
sical  design  of  different  computers.)  As  a  more  prosaic  example,  an 
electric  fuel  pump  In  an  automobile  participates  In  both  the  electrical 
system  and  the  fuel  system  of  the  car.  Presumably  all  of  these  overlap¬ 
ping  organizations  are  accessible  to  the  subject;  which  Is  employed  at  a 
particular  time  depends  on  the  dictates  of  the  task.  The  processing 
demands  that  the  subject  places  on  the  mental  representation  enables  one 
form  or  the  other. 

Thus,  the  cognitive  representation  of  Information  given  In  a  circuit  di¬ 
agram  may  be  usefully  described  as  a  set  of  parallel  networks.  In  part, 
these  networks  have  a  hierarchical  structure.  In  that  the  terminal  nodes 
represent  Individual  circuit  components  (resisters,  capacitors,  etc.), 
the  Intermediate  nodes  represent  either  functional  units  (rectifiers, 
amplifiers,  etc.)  or  physically  proximal  collections  of  components,  and 
the  highest  nodes  represent  the  total  circuit  (a  power  supply,  an  In¬ 
verter,  etc.). 

Clearly,  particular  tasks  use  knowledge  In  different  ways.  Although  the 
range  of  uses  forms  a  complex  and  multidimensional  space,  a  good  case 
can  be  made  for  classifying  knowledge  about  a  complicated  mechanism 
(such  as  a  circuit  diagram)  Into  three  general  classes.  The  definitions 
of  these  classes  follows. 


Structural  and  Functional  Knowledge.  Knowledge  of  this  type  deals  with 
the  way  In  which  a  device  In  constructed  and  the  role  that  Its  parts 
play  In  the  operation.  One  may  know,  for  example,  that  a  transformer 
serves  to  change  the  voltage  of  an  AC  supply,  that  a  particular  combina¬ 
tion  of  transitors  acts  as  a  flip-flop,  and  so  forth.  Fundamentally, 
this  knowledge  Is  static;  It  describes  the  way  that  the  device  Is  put 
together  and  how  It  works.  Information  of  this  type  Is  easiest  to  model 
as  some  form  of  connected  network  (Anderson,  1976;  Anderson  &  Bower, 
1973;  Collins  £  Loftus,  1975;  Norman  &  Rumelhart,  1975;  Schank,  1972). 

In  such  a  model.  Individual  circuit  components  would  be  represented  by 
nodes  in  the  network  and  the  Interrelationships  among  components,  by 
links  between  the  nodes.  In  their  basic  form,  representations  of  this 
sort  are  Isomorphic  to  the  actual  circuit  diagrams.  At  a  higher  level, 
they  also  permit  abstract  nodes  to  represent  the  hierarchical  relation¬ 
ship  among  functional  groupings  of  components.  Host  of  the  work  with 
these  models  has  been  done  In  building  and  testing  theories  of  the  or¬ 
ganization  of  long-term  memory  and  thus  has  been  outside  the  context  of 
problem-solving  tasks.  An  exception  Is  the  work  of  Bhaskar  and  Simon 
(1977)  who  have  undertaken  an  analysis  of  the  structure  of  long-term 
memory  used  by  students  solving  problems  in  a  college-level  course  In 
chemical -engineering  thermodynamics. 

Prototypical  Knowledge.  Devices  are  not  understood  In  Isolation,  but 
are  related  to  other  devices.  Prototypical  knowledge  relates  one  device 
to  more  general  prototypes.  For  example,  experienced  technicians  are 
able  to  quickly  recognize  that  several  different  circuit  diagrams 
represent  the  same  class  of  device.  This  suggests  a  set  of  procedures 
that  force  the  constituent  elements  of  a  circuit  diagram  into  prototypi¬ 
cal  configurations.  One  portion  of  a  circuit  Is  a  rectifier,  another  a 
Schmitt  trigger,  etc.  Two  cognitive  processes  seem  to  be  Involved  here. 
First,  some  form  of  bottom-up  mechanism  simplifies  the  representation  of 
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the  diagram  by  replacing  groups  of  nodes  with  a  single  node.  Second,  a 
top-down  mechanism  attempts  to  fill  in  missing  nodes  in  partially 
matched  prototypes.  Prototypical  knowledge  does  not  consist  of  an  exact 
memory  for  a  particular  circuit;  Instead  it  emphasizes  the  common  ele¬ 
ments  and  patterns  across  a  range  of  devices.  Since  the  operation  of 
any  device  is  related  to  to  an  abstracted  prototype,  the  mental  burden 
of  the  structural  and  functional  facts  is  greatly  reduced. 

Procedural  Knowledge.  Many  tasks  require  more  than  a  static  comprehen¬ 
sion  of  a  device,  demanding  that  some  modification  be  made  or  some 
operation  be  performed  on  It.  The  knowledge  necessary  to  do  this  Is 
different  from  the  other  two  types,  embodying  a  series  of  procedures 
(algorithmic  or  heuristic)  for  changing  the  device.  Procedural 
knowledge  is  the  most  complex  of  the  three  types  of  knowledge,  and  draws 
heavily  on  the  others. 

Because  of  its  active  representation,  procedural  knowledge  has  been 
found  most  useful  in  simulation  (e.g.,  Winograd,  1972)  and  problem  solv¬ 
ing  models  (Newell  &  Simon,  1972);  and  some  models  (e.g.,  Anderson, 

1976)  have  incorporated  procedural  and  other  forms  of  representation  as 
well.  DeKleer  (1979)  and  deKleer  and  Brown  (1980)  have  described,  from 
an  artificial  Intelligence  point  of  view,  some  of  the  procedural  stra¬ 
tegies  that  are  required  to  analyze  the  operation  of  a  circuit.  In  par¬ 
ticular,  they  emphasize  the  need  for  multiple  procedural  strategies. 

For  example,  DeKleer  (1979)  hypothesizes  that  people  use  topological, 

functional,  and  geometric  representations.  In  topological  analysis,  the 
topology  of  a  new  circuit  is  compared  to  that  of  previously  recognized 
circuits;  in  functional  analysis,  the  behavior  of  the  indivdual  com¬ 
ponents;  and  geometric  analysis  relies  on  the  tacit  graphical  language 
engineers  use  when  they  describe  circuit  topologies  on  paper.  These 
representations  are  used  to  analyze  a  circuit  in  terms  of  its  "teleolo- 
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gy."  Similarly,  Stevens  and  Collins  (1980)  argue  that  people  maintain 
multiple  representations  of  physical  systems  such  as  those  that  deter¬ 
mine  the  rainfall. 

Experts  and  Novices 

Previous  work  relevant  to  understanding  how  people  represent  complex 
equipment  from  schematic  drawings  comes  from  research  comparing  the  per¬ 
formance  of  experts  to  that  of  novices.  Considerable  research  In  this 
area  has  Indicated  that  experts  differ  from  novices  more  In  perceptual- 
memorlal  abilities  than  In  logical,  problem-solving  abilities.  If  It 
were  simply  the  case  that  experts  know  more  about  the  task  than  novices, 
there  wou’d  be  little  to  be  gained  from  constructing  an  elaborate 
representation  of  the  task.  However,  this  seems  not  to  be  the  case. 
Experts  appear  to  have  representations  that  differ  qualitatively  from 
those  of  novices,  and  studies  supporting  this  position  are  becoming  more 
common. 

For  example,  Egan  and  Schwartz  (1979)  demonstrated  that  expert  electron¬ 
ic  troubleshooters  have  a  richer  mental  representation  of  circuit  Infor¬ 
mation  than  novice  troubleshooters,  and  In  reconstruction,  the  experts 
recall  the  diagrams  In  groupings  of  functional  units.  The  skilled 
technician's  advantage  In  this  task  did  not  hold  for  non-meanlngfully 
arranged  symbols. 

Work  by  deGroot  (1966),  Chase  and  Simon  (1973),  and  Simon  and  Chase 
(1973),  comparing  the  performance  of  Master  and  weaker  chess  players. 
Indicates  that  Masters  do  not  "see"  ahead  further  than  the  weaker 
players.  Instead,  the  Masters  are  superior  to  weaker  players  In  their 
ability  to  perform  tasks  Involving  the  recall  of  actual  chess  positions. 
The  superior  performance  of  Masters  In  these  tasks  cannot  be  attributed 
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to  generally  superior  visual  short-term  memory  capacities  of  the  Masters 
because  when  hess  pieces  are  placed  randomly  on  the  board,  recall  Is 
equally  poor  for  Masters  and  weaker  players. 

Badre  (1979)  has  found  similar  results  for  the  recall  of  battlefield  si¬ 
tuation  displays.  Military  experts  show  a  marked  advantage  over  novices 
for  plausible  situation  displays,  but  not  for  randomly  arranged  posi¬ 
tions.  Furthermore,  military  experts  recall  battlefield  units  on  the 
basis  of  their  functional  relationship  to  each  other.  Similar  results 
hold  for  other  spatial  tasks,  such  as  the  recall  of  60  positions  (Relt- 
man,  1976),  and  have  also  been  extended  to  other  non-spatlal  tasks  such 
as  the  recall  of  computer  programs  (Reltman,  McKelthen,  Reuter,  S  Hlr- 
tle,  1979)  and  the  solution  of  physics  problems  (Chi,  Feltovlch,  & 
Glaser,  1979;  Larkin,  1979;  Larkin,  McDermott,  Simon,  S  Simon,  1980). 

The  most  appealing  theoretical  explanation  of  these  findings  Is  that  ex¬ 
perts  perceive  spatial  stimuli  by  coding  the  stimuli  Into  groups  con¬ 
sisting  of  several  elements  or  chunks.  In  one  version  of  this  theory 
(Simon  &  Gllmartln,  1973),  the  chunks  have  verbal  labels  that  can  be  re¬ 
tained  In  short-term  memory  and  decoded  at  the  time  of  recall.  It  Is 
argued  that  experts  quickly  represent  an  entire  spatial  configuration  In 
a  relatively  small  number  of  chunk  labels  and  that  these  labels  can  be 
used  to  reconstruct  the  spatial  configuration.  Pauses  between  succes¬ 
sively  recalled  elements,  the  estimated  size  and  number  of  chunks,  and 
the  correspondence  of  recall  groupings  In  copying  tasks  support  this  hy¬ 
pothesis. 

However,  the  difference  between  experts  and  novices  may  lie  as  much  or 
more  In  the  ability  of  experts  to  draw  from  a  larger  collection  of 
operations— or  procedural  knowledge— than  In  an  understanding  of  the 
structural  and  functional  nature  of  the  parts  of  the  task  or  In  the 
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reference  to  prototypes.  Much  previous  work  has  Ignored  the  dynamic, 
procedural  aspects  of  complex  tasks  In  favor  of  more  static  conceptual  1 
zatlons. 


Research  Perspective 

Reviews  of  the  literature  on  the  mental  representation  of  tasks  or 
stimuli  reveal  that  the  derived  representations  are  surprisingly  well 
matched  to  the  task.  This  suggests  that  a  person  Is  able  to  adopt  a 
mental  representation  that  Is  closely  concordant  with  local  processing 
demands.  The  true  mental  representation  must  have  latent  In  It  a 
variety  of  possible  forms  and  structures.  Consequently,  any  analysis  of 
the  mental  characteristics  of  a  domain  of  knowledge  must  derive  from  a 
variety  of  tasks  and  must  posit  a  variety  of  Individual  representations. 
Also,  the  understanding  of  skilled  performance  In  tasks  such  as  circuit 
analysis  requires  consideration  of  multiple  domains  of  knowledge.  That 
1$,  different  domains  are  necessary  to  explain  the  richness  of  an  expert 
troubleshooter's  mental  constructs,  and  a  comprehensive  model  must  In¬ 
corporate  elements  of  each  domain. 

The  experiment  reported  here  was  designed  to  assess  the  structural  and 
functional  knowledge  of  electronic  devices  possessed  by  technicians 
varying  in  skill  level.  This  work  provides  the  necessary  background  for 
the  future  Investigation  of  procedural  knowledge  as  related  to  electron¬ 
ic  trouble-shooting.  The  experiment  performed  was  a  composite  of  a 
circuit-reconstruction  task  and  a  circuit-partitioning  task. 
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METHOD 


Materials 


Three  circuit  diagrams  Mere  extracted  from  manuals  of  Navy  electronic 
equipment  for  use  In  the  experiment.  Each  diagram  was  sufficiently 
large  to  represent  several  separate  subsections  and  to  contain  enough 
nodes  to  have  a  reasonably  complex  Internal  structure.  The  diagrams 
were  selected  to  represent  three  levels  of  complexity. 

The  small  circuit,  Mhlch  contains  25  component  parts.  Is  shoMn  In  Figure 
1.  This  circuit  consists  of  a  simple  tMO-stage  amplifier  for  a  video 
signal.  It  contains  tMo  amplifier  stages,  centered  on  the  tMo  transit* 
tors  Mlth  their  associated  circuitry.  There  Is  a  single  supply  voltage, 
vcc*  c1rcu1t  w*s  used  with  the  less  skilled  subjects  only. 

The  medium-sized  circuit,  Mhlch  contains  32  component  parts.  Is  shoMn  In 
Figure  2.  This  circuit,  Mhlch  Is  part  of  a  radio  receiver,  generates  a 
radio  frequency  signal,  and  then  mixes  (l.e.,  heterodynes)  It  Mlth 
another  signal  that  enters  from  the  previous  stages  on  the  Input  line. 
The  RF  signal  Is  generated  by  an  oscillator,  composed  primarily  of  the 
transistor  and  a  crystal.  This  signal  Is  coupled  through  a  transformer 
to  the  mixer  Itself.  The  combined  result  Is  fed  to  the  next  stage  of 
the  receiver.  A  single  16-volt  supply  poMers  the  circuit.  This  circuit 
Mas  used  Mlth  all  subjects. 

The  large  circuit  Is  presented  In  Figure  3.  This  circuit  consists  of 
the  final  audio  stages  of  a  radio  receiver  and  contains  84  component 
parts.  The  signal  from  the  radio's  discriminator  enters  to  the  audio 
preamplifier.  It  passes  from  there  through  a  1oM-pass  filter  to  remove 
high-frequency  noise,  then  through  the  final  poMer  amplifier  to  the 
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speaker.  At  the  same  time,  the  unfiltered  signal  Is  fed  to  another  am¬ 
plifier  that  contains  a  feedback  network, to  make  It  maximally  sensitive 
to  Interstation  noise.  In  the  presence  of  this  noise,  the  signal  Is 
rectified  and  amplified  to  throw  the  squelch  relay,  disabling  the  signal 
to  the  power  amplifier  and  suppressing  the  output.  There  are  two  power 
supply  voltges  for  the  circuit,  with  positive  voltages  of  16  and  25. 5 
volts.  This  circuit  was  used  only  with  the  expert  subjects. 


Each  diagram  was  prepared  In  five  configurations  and  three  formats.  The 
five  configurations  of  each  diagram  differed  only  In  the  layout  on  the 
page.  Use  of  these  different  configurations  throughout  the  experiment 
was  to  prevent  the  subject  from  learning  only  properties  of  the  diagram 
specific  to  the  physical  layout  of  the  diagram.  The  three  formats  were 
constructed  giving  successively  less  and  less  Information  about  the  par¬ 
ticular  components  while  constraining  the  basic  orientation  of  the  cir¬ 
cuit.  The  formats  were  (1)  the  complete  diagram  as  in  Figures  1,  2,  and 
3;  (2)  a  diagram  showing  Interconnections  in  which  the  particular  com¬ 
ponents  have  been  replaced  by  black  dots;  and  (3)  a  diagram  showing  only 
the  connections,  but  lacking  specific  Indications  of  where  the  com¬ 
ponents  are  located.  Examples  of  the  reduced  formats,  for  the  confi¬ 
guration  of  the  medium-sized  circuit  shown  In  Figure  2  are  given  In  Fig¬ 
ures  4  and  5.  Note  that  both  the  power-supply  voltages  and  the  ground 
points  are  still  shown  In  these  figures.  These  points  were  not  part  of 
either  the  reconstruction  or  the  recall  task. 

Each  of  the  diagrams  was  drawn  on  a  piece  of  paper  roughly  2'  x  3',  such 
that  they  could  then  be  placed  on  a  magnetic  board  of  the  same  size. 
Magnetized  components  matching  the  components  on  the  diagram  were  con¬ 
structed  at  about  3/4“  In  size. 
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FIGURE  5. 

THE  MEDIUM-SIZED  CIRCUIT  OF  FIGURE  2 
(CONNECTIONS-ONLY  FORMAT). 


Procedure 


Each  subject  participated  In  the  experiment  Individually.  The  procedure 
was  divided  Into  several  phases,  which  were  carried  out  In  either  two  or 
three  sessions  depending  upon  the  expertise  of  the  subject.  The  profi¬ 
cient  subjects  worked  with  the  small  and  medium  sized  circuits,  whereas 
the  expert  subjects  worked  with  the  medium-sized  and  large  circuits. 
Trials  on  the  two  circuits  on  which  a  given  subject  was  working  (small 
and  medium,  or  medium  and  large)  were  alternated,  so  that  two  tasks  were 
in  progress  simultaneously.  Each  task  thus  provided  short-term  memory 
interference  for  the  other. 

Learning  the  Diagrams.  In  the  learning  or  acquisition  phase  of  the  ex¬ 
periment,  each  subject  was  asked  to  reconstruct  several  circuit  layouts 
for  each  of  the  two  circuits  to  be  worked  on  by  that  subject.  To  facil¬ 
itate  learning,  these  reconstructions  were  carried  out  from  successively 
more  and  more  Impoverished  versions  of  the  diagrams.  The  subject  first 
was  presented  with  one  layout  of  a  diagram.  In  its  complete  format  and 
was  given  a  brief  explanation  of  the  circuits  function  and  operation. 

The  descriptions  of  the  circuits  presented  In  the  Materials  section 
above  were  read  to  the  subject  while  the  experimenter  pointed  to  the 
particular  parts  of  the  diagram  as  they  were  mentioned.  Then,  with  the 
complete  diagram  In  view,  the  subject  duplicated  the  circuit  by  placing 
the  magnetic  components  on  the  lines-only  format  of  the  diagram  In  the 
Identical  layout.  This  procedure  was  completed  for  two  different  lay¬ 
outs  of  each  of  the  two  circuits,  alternating  between  the  circuits  with 
which  the  subject  was  working. 

Following  this  Initial  familiarization  procedure,  the  complete  diagrams 
were  removed  from  view  and  the  subject  was  shown  a  dot  format  of  one  of 
the  two  circuits.  The  subject  was  asked  to  reconstruct  the  circuit  from 
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memory  by  placing  the  magnetic  components  on  the  layout.  More  com¬ 
ponents  of  each  type  were  available  than  actually  needed.  When  the  sub¬ 
ject  finished,  the  experimenter  recorded  any  errors  of  placement  or  om¬ 
issions.  Then,  the  subject  was  given  a  copy  of  the  complete  circuit  and 
was  instructed  to  correct  any  errors.  This  procedure  was  repeated,  with 
alternating  trials  on  each  of  the  two  circuits,  until  two  different  lay¬ 
outs  had  been  completed. 

Finally,  the  subject  was  asked  to  reconstruct  each  of  the  two  circuits 
from  memory  using  layouts  that  showed  only  the  circuit  connections,  thus 
lacking  specific  Indications  of  where  the  components  were  located.  This 
portion  of  the  procedure  was  repeated,  using  different  layouts  and  al¬ 
ternating  between  circuits,  until  two  completely  correct  reconstructions 
were  achieved  for  each  circuit.  Errors  In  component  placement  and  the 
number  of  trials  to  learn  each  circuit  were  recorded. 

First  Component  Partitioning  Task.  After  the  Initial  acquisition  phase 
of  the  experiment  was  completed,  the  subject  was  given  a  copy  of  the  di¬ 
agram  In  a  layout  not  seen  before  and  a  set  of  colored  pencils.  The 
subject  was  asked  to  circle  and  identify  the  circuit  components  that  go 
together,  using  different  colors  to  indicate  the  degree  to  which  com¬ 
ponents  went  together  (see  Table  1  for  the  colors  and  verbal  codes). 

The  task  was  introduced  by  an  example  which  Illustrated  (1)  the  use  of 
the  different  colors  to  designate  various  degrees  of  component  related¬ 
ness,  and  (2)  that  a  single  component  can  participate  in  several  dif¬ 
ferent  clusters  at  the  same  time,  not  necessarily  overlapping.  The  data 
collected  in  this  phase  consisted  of  the  participants'  partitionings  of 
the  diagrams. 

Reconstruction  Trials.  The  general  procedure  in  the  reconstruction 
phase  of  the  experiment  was  to  have  the  subject  perform  five  additional 


TABLE  1 

COLOR  CODED  SCALE  FOR  USE  IN 
THE  PARTITIONING  TASK 


SCALE  DESCRIPTIVE  COLOR 

VALUE  PHRASE  CODE 


7  Inseparably  related  Violet 

6  Blue 

5  Green 

4  Moderately  related  Yellow 

3  Orange 

2  Red 

1  Weakly  related  Brown 
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reconstructions  of  the  two  circuits.  The  connections-only  layouts  were 
rotated  across  trials,  alternating  between  the  two  circuits.  This  time, 
the  experimenter  recorded  the  order  in  which  the  components  were  placed 
on  each  layout. 

To  achieve  a  variety  of  orders,  four  of  the  five  trials  on  each  circuit 
were  begun  with  the  experimenter  placing  one  component  on  the  layout. 

On  these  cued  reconstruction  trials,  the  subject  was  told: 

"I’m  going  to  put  one  piece  down  first.  You  should 
reconstruct  the  circuit  as  before,  but  start  by  building 
out  from  the  piece  I  put  down.  You  do  not  have  to  make 
every  piece  connect  to  that  one,  but  generally  you 
should  start  building  from  that  part  of  the  diagram  and 
complete  that  part  first." 

For  each  circuit,  two  different  cue  locations  were  selected  and  they 
were  used  twice  during  the  experiment.  These  components  are  designated 
by  asterisks  in  Figures  1,  2,  and  3. 

Second  Component-Partitioning  Task.  At  the  end  of  the  series  of  recon¬ 
structions,  the  component  partitioning  task  was  repeated,  exactly  as  be¬ 
fore.  This  repeat  procedure  was  included  to  measure  the  extent  to  which 
the  understanding  of  tne  diagram  had  changed  during  the  reconstruction 
phase. 

Subjects 

Eight  subjects  participated  in  this  experiment,  of  which  seven  were 
classified  as  proficients.  All  subjects  had  recently  obtained  a  B.S. 
degree  in  electrical  engineering  with  at  least  one  course  in  analog  ra¬ 
dio  circuits  and  some  post-bachelor  experience  with  electrical  circuits. 
Four  of  these  subjects  were  enrolled  as  graduate  students  in  electrical 
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engineering  during  the  time  of  the  experiment,  and  each  had  taken  an  ad¬ 
vanced  class  In  circuit  theory.  The  subject  that  was  classified  as  an 
expert  had  both  technical  training  (the  equivalent  of  a  Master's  degree 
In  electrical  engineering)  and  an  appreciable  amount  of  actual  experi¬ 
ence  In  designing  and  maintaining  electronic  equipment.  The  subjects 
were  first  classified  by  experience:  the  one  advanced  subject  was  con¬ 
sidered  an  expert;  and  the  other  seven  were  considered  proficients.  The 
proficient  subjects  were  further  classified  as  high  or  low  proficient  on 
the  basis  of  two  performance  crite,  a,  total  time  to  complete  the  ex¬ 
periment  and  total  number  of  errors  made  during  the  experiment.  These 
data  are  presented  In  Table  2.  The  total  test  time  varied  as  a  function 
of  both  how  long  the  subject  took  to  learn  the  circuits  and  to  recon¬ 
struct  the  circuit  layouts. 

It  is  worthy  of  note  In  Table  2  that  the  error  rates  for  high  and  low 
proficients  differ  more  sharply  with  the  medium-sized  circuit  that  with 
the  small  circuit.  This  underscores  the  need  to  use  sufficiently  com¬ 
plex  circuits  In  tasks  of  the  type  studied  here.  Otherwise,  differences 
between  skill  levels  may  be  obscured. 

Method  of  Analysis 

Reconstruction  Data.  The  multitrial  free  recall  data  were  analyzed  us¬ 
ing  Reitman  and  Reuter's  (1980)  algorithm  for  creating  "ordered  trees." 
The  design  of  the  experiment  was  such  as  to  give  about  six  complete  re¬ 
call  orders  for  each  subject  (Including  the  two  recalls  that  preceded 
the  first  circling  task),  of  which  four  were  cued  and  two  were  free. 
These  six  orders  were  analyzed  using  the  Reltman-Reuter  procedure  and 
their  structure  extracted.  This  clustering  procedure  yields  a  tree-like 
rendition  of  a  subject's  mental  representation  of  a  knowledge  base.  In 
such  a  tree,  the  branches  represent  Individual  elements  and  the  nodes 
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TABLE  2 

TIME  TO  COMPLETE  EXPERIMENT  AND  NUMBER  OF  ERRORS 
MADE  LEARNING  CIRCUIT  FOR  INDIVIDUAL  SUBJECTS 


ERRORS 


SMALL 

MEDIUM 

LARGE 

GROUP 

SUBJECT 

TIME 

CIRCUIT 

CIRCUIT 

CIRCUIT 

Low 

Proficients 

B.B. 

5.00  hrs 

3 

32 

J.S. 

8.25  hrs 

15 

109 

- 

B.N. 

6.00  hrs 

3 

17 

- 

C.S. 

4.75  hrs 

0 

27 

- 

Average 

6.00  hrs 

5.25 

46.25 

- 

High 

Proficients 

D.T. 

4.00  hrs 

4 

3 

- 

D.R. 

4.25  hrs 

0 

11 

- 

I.K. 

3.50  hrs 

0 

3 

m 

m 

Average 

3.91  hrs 

1.33 

5.67 

- 

Expert 


M.E 


7.15  hrs 


6 
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represent  clusters  or  chunks  of  elements  that  “go  together."  Generally, 
the  closer  a  node  is  to  the  root  of  the  tree,  the  less  related  are  the 
elements  which  combine  at  that  node. 

The  resulting  trees  in  the  present  research  can  be  Illustrated  In  two 
ways:  first,  as  the  recall  tree  Itself;  and  second,  as  a  set  of  clus¬ 
ters  that  represent  the  nodes  of  the  tree  drawn  on  the  circuit  Itself. 
Because  the  number  of  different  recall  orders  that  are  used  as  Input  to 
the  program  Influences  the  amount  of  complexity  that  is  derived,  it  Is 
Important  to  compare  these  statistics  primarily  for  subjects  for  which 
the  same  amount  of  data  has  been  obtained. 

Once  the  tree  has  been  constructed,  a  number  of  quantitative  measures 
can  be  derived  from  It.  The  most  simple  of  these  are  the  number  of 
clusters  In  the  tree  and  the  depth  to  which  these  clusters  are  nested. 
Since  the  algorithm  designates  each  cluster  as  recalled  in  a  fixed  or  an 
arbitrary  order,  the  proportion  of  clusters  containing  three  or  more 
elements  that  are  recalled  directionally  gives  a  further  Indication  of 
strength.  A  more  comprehensive  measure  Is  obtained  by  calculating  the 
number  of  possible  recall  orders  (PROs)  that  are  consistent  with  the 
tree  as  derived.  For  a  tree  of  any  substantial  size,  the  PRO  is  quite 
large  and  so  It  Is  somewhat  easier  to  interpret  the  logarithm  of  the 
PRO.  The  base-two  logarithm  Is  used  here,  so  that  the  results  can  be 
Interpreted  as  a  measure  of  Information  uncertainty  (e.g..  Shannon  & 
Weaver,  1949).  Large  values  of  this  statistic  Indicate  that  the  recall 
orders  show  little  consistent  patterning,  while  small  values  Indicate  a 
large  degree  of  consistency. 

Partitioning  Data.  The  analysis  strategy  was  to  use  the  ADCLUS  model  of 
Arable  and  Carroll  (1980)  to  covert  the  partitionings  Into  a  set  of 
proximity  matrices,  and  then  to  use  various  other  scaling  techniques  to 
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reconstruct  a  pooled  order  from  these.  The  circles  marked  by  the  sub¬ 
jects  on  the  diagrams  were  converted  to  proximity  matrices  by  a  direct 
application  of  the  ADCLUS-type  weighting  model.  For  each  pair  of  com¬ 
ponents,  the  proximity  between  the  components  was  calculated  by  summing 
the  weights  associated  with  all  circles  that  contained  both  components. 
Thus,  if  two  components  were  contained  both  in  a  red  and  a  green  circle 
(weights  2  and  5,  respectively),  then  a  proximity  of  2  +  5  ■  7  was  as¬ 
signed  to  the  pair.  By  averaging  over  any  set  of  partitioning,  a  pooled 
proximity  matrix  was  obtained.  In  particular,  the  circling  data  for  the 
subjects  at  various  levels  of  skill  were  combined  Into  a  single  proximi¬ 
ty  matrix. 

Because  the  underlying  structure  of  the  mental  representation  was  ex¬ 
pected  to  involve  a  number  of  parallel  hierarchies,  It  made  most  sense 
to  analyze  the  partitioning  data  using  a  procedure  that  allows  overlap¬ 
ping  clusters  to  be  extracted.  Programs  for  this  type  of  clustering  ex¬ 
ist  and  they  were  applied  here  In  both  two-mode  and  a  three-mode  manner 
(e.g.,  the  ADCLUS  procedure  In  Sarle,  1981). 

RESULTS 

The  results  are  presented  In  terms  of  errors  made  during  the  learning 
phase  of  the  experiment,  recall  orders  from  the  reconstruction  tasks, 
and  partitioning  clusters  from  the  partitioning  task. 

Errors  During  Acquisition 

The  average  number  of  errors  made  by  the  subjects  In  the  Initial  recon¬ 
struction  and  learning  of  the  circuits  is  shown  In  Table  3.  Errors  are 
categorized  by  the  type  of  circuit  element  and  they  are  averaged  over 
ability  group.  Three  characteristics  are  apparent  in  the  data.  First, 
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TABLE  3 

MEAN  ERRORS  DURING  LEARNING  OF  CIRCUIT 


CIRCUIT  COMPONENT 


Resistor 
_  (R)  . 

Capacitor 

(O 

Transistor 
or  Diode 
(Q)  _ 

Inductor 

(L) 

TOTAL 

SMALL  CIRCUIT 

Low  Proficients 

1.8 

1.2 

0.0 

2.2 

5.2 

High  Proficients 

0.7 

0.7 

0.0 

0.0 

1.3 

MEDIUM  CIRCUIT 

Low  Proficients 

11.5 

18.2 

7.8 

10.8 

48.2 

High  Proficients 

1.7 

2.0 

0.7 

1.3 

5.7 

Expert 

1 

4 

0 

1 

6 

LARGE  CIRCUIT 

Expert 

9 

0 

5 

0 

11 
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the  number  of  errors  Is  considerably  smaller  for  the  more  experienced 
subjects,  as  would  be  expected.  Second,  the  proportional  distribution 
by  circuit  component  type  is  similar  at  all  levels  of  circuit  difficul¬ 
ty.  Finally,  the  number  of  errors  In  learning  the  circuit  Is  very  low 
for  the  best  qualified  subjects.  Diagrams  of  this  complexity  are  not 
hard  to  learn  for  subjects  familiar  with  the  circuits.  However,  this 
high-level  of  performance  makes  It  difficult  to  Identify  response  pat¬ 
terns  for  these  subjects. 

Recall  Orders 


Several  examples  of  trees  generated  from  data  for  the  small  and  medium 
circuits,  are  shown  In  Figures  7  and  8  and  10  through  12.  These  trees, 
derived  from  subjects  with  different  degrees  of  ability,  were  chosen  to 
Illustrate  the  wide  range  of  behavior  that  was  observed.  Labels  for  the 
components  of  the  small  circuit  are  provided  In  Figure  6.  Figure  7 
shows  the  tree-structure  results  from  a  low  proficient  subject  on  the 
small  circuit.  This  tree  shows  less  order  than  that  from  any  other  sub¬ 
ject,  with  only  one  cluster  appearing  consistently  across  reconstruc¬ 
tions.  For  this  same  circuit.  Figure  8  shows  a  more  proficient  subject 
who  provides  a  much  higher  degree  of  order.  The  corresponding  diagrams 
for  these  subjects  on  the  medium  circuits  are  presented  In  Figures  10 
and  11.  Labels  for  the  components  of  the  medium  circuit  are  provided  In 
Figure  9.  Here,  both  subjects  have  a  fairly  high  degree  of  order.  The 
final  diagram  (Figure  12)  shows  the  clustering  generated  by  the  expert 
subject  on  the  medium  circuit.  Most  striking  In  Figures  6-12  Is  the 
high  degree  of  Inconsistency  across  the  set  of  trees.  Although  the 
largest  degree  of  within  subject  variability  and  of  Inconsistency  across 
skill  ranges  Is  Illustrated  here,  results  from  the  other  subjects  are 
similar. 
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FIGURE  7. 

TREE  DIAGRAM  OF  THE  REITMAN-REUTER  SOLUTION 
FOR  A  LOW-PROFICIENT  SUBJECT  ON  THE  SMALL 
CIRCUIT  (THE  COMPONENT  LABELS  ARE  PROVIDED 
IN  FIGURE  6). 

[Nodes  «  2;  PRO  -  6.20  x  1023;  Log2  PRO  ■  79.04] 


FIGURE  8. 

TREE  DIAGRAM  OF  THE  RE ITMAN- REUTER  SOLUTION 
FOR  A  HIGH-PROFICIENT  SUBJECT  ON  THE  SMALL 
CIRCUIT  (THE  COMPONENT  LABELS  ARE  PROVIDED 
IN  FIGURE  6). 

[Nodes  •  11;  PRO  -  3.01  x  1011;  Log2  PRO  -  38.13] 
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FIGURE  10. 

TREE  0IAGRAM  OF  THE  RE I THAN- REUTER 
SOLUTION  FOR  A  LOW-PROFICIENT  SUBJECT 
ON  THE  MEDIUM  CURCUIT  (THE  COMPONENT 
LABELS  ARE  PROVIDED  IN  FIGURE  9). 

[Nodes  «  18;  PRO  «  2,58  x  106;  Log2  PRO  -  21.30] 
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FIGURE  11. 

TREE  DIAGRAM  OF  THE  REITMAN-REUTER 
SOLUTION  FOR  A  HIGH-PROFICIENT  SUBJECT 
ON  THE  MEDIUM  CIRCUIT  (THE  COMPONENT 
LABELS  ARE  PROVIDED  IN  FIGURE  9). 

[Nodes  ■  18;  PRO  «  1.475  x  106;  Log2  PRO  ■  20.49] 
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FIGURE  12. 

TREE  DIAGRAM  OF  THE  RE I THAN- REUTER 
SOLUTION  FOR  THE  EXPERT  SUBJECT  ON 
THE  MEDIUM  CIRCUIT  (THE  COMPONENT 
LABELS  ARE  PROVIDED  IN  FIGURE  9). 

[Nodes  »  11;  PRO  -  9.216  x  104;  Log2  PRO  -  16.477] 
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TABLE  4 

SUMMARY  STATISTICS  FOR  TREES  RECOVERED 
BY  REITMAN-REUTER  ALGORITHMS 


Low-Proficient  Subjects 

Hiqh-Proficient  Subjects 

Measure 

Range 

Mean 

Range 

Mean 

SMALL  CIRCUIT 

No.  of  Clusters 

1-10 

4.2 

5-10 

7.0 

Average  Size 

2. 0-2. 8 

2.4 

2. 0-3. 6 

2.6 

Maximum  Levels 

1-2 

1.5 

1-3 

1.5 

i  ™  PR0 

Log2 

31.0-79.0 

65.3 

38.1-64.1 

54.65 

MEDIUM  CIRCUIT 

No.  of  Clusters 

8-17 

11.8 

10-17 

12.7 

Average  Size 

2. 3-5. 4 

3.4 

2. 4-4.1 

3.0 

Maximum  Levels 

2-5 

3.0 

2-4 

2.7 

i  nn  PR0 

Log2 

21.3-72.7 

52.4 

20.5-74.0 

53.1 
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Summary  statistics  for  the  trees  are  shown  In  Table  4;  again  most  strik¬ 
ing  is  the  degree  of  similarity  In  average  performance  between  the  high 
and  low-proficient  groups.  No  differences  among  the  measures  reached 
statistical  significance.  While  It  Is  possible  that  with  a  considerably 
larger  group  of  subjects,  some  reliable  differences  would  be  found,  it 
Is  apparent  that  they  would  be  relatively  small  in  comparison  with  the 
variability  in  performance  within  the  skill  levels. 

Partition  Clusters.  The  data  obtained  in  the  partitioning  task  were 
first  pooled  over  both  clrcllngs  for  each  subject  and  for  all  subjects 
within  an  ability  level,  creating  two  matrices  for  each  diagram.  These 
were  first  scaled  alone,  and  then  in  a  three-mode  manner  to  extract  com¬ 
mon  clusters.  These  clusterings  are  shown  In  Figures  13-15  and  16-18 
for  the  small  and  medium  circuits,  respectively.  The  first  figure  in 
each  set  shows  the  clusters  recovered  from  the  data  of  the  low- 
proficient  subjects.  The  second  shows  the  same  data  for  the  high- 
proficient  subjects,  and  the  last  shows  the  results  of  a  three-mode 
analysis  based  on  both  sets  of  data.  Each  cluster  is  labeled  with  the 
percentage  of  the  total  variance  of  the  proximities  that  could  be  ac¬ 
counted  for  by  that  cluster. 

Figure  13  shows  the  four  clusters  that  accounted  for  the  largest  amount 
of  variance  (73%  in  all).  Two  clusters  correspond  to  the  two  amplifier 
stages  of  the  circuit,  the  other  two  clusters  block  all  except  the  out¬ 
put  portions  of  the  same  groups.  The  full  clustering  solution  for  this 
diagram  contained  other  clusters  as  well,  but  these  accounted  for  rela¬ 
tively  small  amounts  of  the  variance  and  they  have  not  been  plotted 
here.  Figure  14  shows  a  similar  diagram  for  the  high-proficient  sub¬ 
jects,  and  three  of  the  clusters  are  similar  to  those  In  Figure  13.  The 
remaining  cluster  in  Figure  14  puts  together  the  output  filters  of  both 
stages,  illustrating  the  clustering  of  components  by  similar  functioning 
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FIGURE  14. 

AOCLUS  SOLUTION  FOR  THE  SMALL  CIRCUIT, 
HIGH-PROFICIENT  SUBJECTS. 
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FIGURE  15. 

ADCLUS  SOLUTION  FOR  SMALL  CIRCUIT 
THREE-MODE  ANALYSIS. 
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FIGURE  16. 

ADCLUS  SOLUTION  FOR  THE  MEDIUM  CIRCUIT 
LOW-PROFICIENT  SUBJECTS. 


FIGURE  17. 

AOCLUS  SOLUTION  FOR  THE  MEDIUM  CIRCUIT, 
HIGH-PROFICIENT  SUBJECTS. 
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AOCLUS  SOLUTION  FOR  THE  MEDIUM  CIRCUIT, 
THREE-MODE  ANALYSIS. 
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in  a  way  that  cuts  across  component  type  or  physical  proximity.  Figure 
15  shows  the  same  data  scaled  in  a  three-mode  manner,  so  that  clusters 
are  selected  that  are  based  on  either  group's  responses.  With  a  few 
differences,  this  solution  roughly  reproduce  the  patterns  that  were  seen 
for  the  individual  group  solutions. 

The  clustering  of  the  medium  circuit  is  illustrated  in  Figures  16-18. 
Overlapping  clusters  are  less  prevalent  here  than  with  the  small  cir¬ 
cuit.  The  most  reliably  clustered  components  are  those  that  were  in¬ 
volved  in  tlte  oscillator  portion  of  the  circuit.  Because  of  the  weight¬ 
ing  of  the  original  clusters,  a  reliably,  but  low-weighted  cluster  can 
have  small  explained  variance  in  the  ultimate  solution.  Several  such 
low-variance  clusters  are  included  in  the  single-group  diagrams. 

The  clusters  in  Figures  13-18  do  not  necessarily  reproduce  clusters  that 
appeared  in  the  clusters  generated  by  individual  subjects.  When  several 
subjects  circle  similar,  but  not  Identical  groups  of  components,  the 
scaling  procedure  generates  clusters  that  show  the  union  and  the  inter¬ 
section  of  the  clusters.  For  example,  if  three  subjects  circled  the 
sets  of  components  (a,b,c,d),  (b,c,d,e),  and  (a.c.d.e),  then  the  full 
set  would  show  one  level  of  proximity,  and  the  two  elements  in  the  in¬ 
tersection  would  be  represented  on  a  higher  level.  The  result  of  this 
scale  would  give  the  double  cluster  [a,b,(c,d) ,e].  The  most  obvious 
place  where  this  phenomenon  appears  to  have  happened  is  with  clusters  2 
and  3  of  Figure  16. 


DISCUSSION 

The  principal  finding  evident  from  the  data  presented  here  Is  the  large 
degree  of  variability  In  the  performance  among  the  subjects  on  the 
reconstruction  and  cluster-generating  tasks,  and  the  extent  of  overlap 


41 


between  the  different  ability  groups  on  those  tasks.  These  data  stand 
In  contrast  to  the  relatively  large  differences  between  skill  levels 
that  appear  in  the  time  and  error  measures  with  respect  to  overall  per¬ 
formance  (Tables  2  and  3).  The  size  of  the  subject  samples  are  too 
small  to  apply  statistical  tests  to  data  such  as  those  summarizing  the 
Reltman-Reuter  trees  in  Table  4,  but  it  Is  clear  that  wlthln-group  vari¬ 
ability  on  those  performance  measures  are  larger  than  the  between-group 
differences.  Even  If  a  large  sample  could  provide  statistically  signi¬ 
ficant  differences  between  skill  levels,  the  size  of  the  effect  as  a 
proportion  of  variability  would  probably  not  be  very  great.  This  con¬ 
clusion  Is  all  the  more  apparent  from  a  comparison  of  the  wlthln-subject 
variation  In  performance  on  the  small  medium  circuits  (Figures  7  and  8 
compared  to  Figures  10  and  11,  respectively,  provide  a  particularly 
striking  example). 

The  Reltman-Reuter  procedure,  as  applied  here.  Is  highly  sensitive  to 
changes  In  recall  order;  that  Is,  where  recall  orders  are  different,  the 
procedure  Infers  no  structure  at  all.  Thus,  the  use  of  circuit  diagrams 
that  were  constrained  by  their  physical  layout  may  have  Imposed  a  varied 
structure  on  recall  order  from  trial  to  trial.  These  constraints,  by 
necessities  of  the  experimental  design,  differed  from  layout  to  layout 
and  may  have  served  to  alter  the  recall  orders  sufficiently  so  as  to 
weaken  the  results.  Although  the  Reltman-Reuter  procedure  Is  potential¬ 
ly  useful  for  analyzing  performance  on  reconstruction  tasks  of  this 
sort.  It  would  probably  be  more  valuable  when  applied  to  situations  In 

which  full  free  recall  Is  used. 

Nevertheless,  there  are  a  few  places  where  performance  differences 
between  different  ability  groups  do.  In  fact,  appear.  First,  as  the 
original  basis  for  the  ski  11 -level  division  shows,  there  are  substantial 
differences  In  the  speed  and  accuracy  measures  for  learning  and  recon- 
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structlng  the  diagrams  (Tables  2  and  3).  Additional ly,  in  some  in¬ 
stances,  differences  in  the  tree  structures  and  clusterings  showed  the 
emergence  of  a  different  type  of  circuit  organization  for  the  better 
subjects  (see,  particularly,  the  results  for  the  partitioning  tasks-- 
Figures  13  through  18).  The  relatively  small  size  of  these  effects  and 
the  large  variance  of  the  measures  suggest,  however,  that  the  investiga¬ 
tion  of  structural /functional  knowledge  with  respect  to  these  type  of 
circuit  diagrams  is  not  the  most  productive  place  for  further  research 
effort. 

Hence,  identifiable,  differences  In  the  organization  of  a  diagram  during 
learning  and  recall  do  not  appear  to  be  the  most  sensitive  loci  of  pro¬ 
ficiency  differences.  For  reasonably  simple  and  well -learned  material, 
the  physical  properties  of  the  diagram  layout  may  dominate  performance. 
It  Is  conceivable  that  differences  could  appear  between  subjects  of  dif¬ 
ferent  proficiency  levels,  with  the  less  proficient  subjects  being  more 
bound  by  the  physical  construction  of  the  target  circuit,  and  the  more 
proficient  subjects  being  more  bound  by  the  logical  construction.  In 
conventionally  well-drawn  circuit  diagrams,  however,  these  two  organiza¬ 
tions  coincide,  minimizing  differences  in  performance. 

In  contrast,  more  substantial  differences  ire  more  likely  to  appear  in 
the  way  in  which  the  circuit  diagrams  are  manipulated,  that  is,  in 
operations  that  are  performed  on  the  diagrams.  Even  with  improperly 
drawn  diagrams,  where  the  logical  and  physical  aspects  conflict,  the 
expert's  advantage  would  come  through  an  ability  to  reorganize  the  cir¬ 
cuit.  Thus,  the  principal  performance  differences  between  the  most 
skilled  and  least  skilled  subjects  may  be  a  result  of  differences  in 
their  respective  levels  of  procedural  knowledge.  For  this  reason,  the 
next  phase  of  this  work  will  examine  performance  on  tasks  that  require 
technicians  to  manipulate  circuits  in  prescribed  ways.  Such  research. 


focusing  on  the  Investigation  of  procedural  knowledge,  will  complement 
tile  present  exploration  of  Individual  differences  In  structural /functional 
knowledge. 


REFERENCES 


Anderson,  J.R.  Language,  memory  and  thought.  Hillsdale,  N.J.:  Erlbaum, 
1976. 

Anderson,  J.R.,  &  Bower,  G.H.  Human  associative  memory.  Washington,  D.C.: 
Winston,  1973. 

Arable,  P.,  4  Carroll,  J.D.  MAPCLUS:  A  mathematical  approach  to  fitting 
the  ADCLUS  model.  Psychometrika,  1980,  45,  211-235. 

Badre,  A.M.  Selecting  and  representing  information  structures  for  battle¬ 
field  decision  systems.  (Report  No.  TR-79-X207J  Alexandria,  V A: 

U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences, 

June  1979. 

Bhaskar,  R.,  4  Simon,  H.A.  Problem  solving  in  semantically  rich  domain: 

An  example  from  engineering  thermodynamics.  Cognitive  Science,  1977, 

1,  139-215. 

Bond.  N.A.,  4  Towne,  D.M.  Troubleshooting  complex  equipment  in  the 
military  services:  Research  and  prospects  (Technical  Report  92).  Los 
Angeles:  University  of  Southern  California,  Behavioral  Technology 
Laboratories.  December  1979. 

Brown,  J.S.,  Collins,  A.,  4  Harris,  G.  Artificial  intelligence  and 
learning  strategies.  In  H.F.  O'Neil,  Jr.  (Ed.),  Learning  Strategies. 

New  York:  Academic  Press,  1978. 

Chase.  W.G.,  4  Simon.  H.A.  Perception  In  chess.  Cognitive  Psychology, 
1973,  4,  55-81. 

Chi,  M.,  Feltovlch,  P.,  4  Glaser,  R.  Physics  problem  solving  by  experts 
and  novices.  Paper  presented  at  the  Twentieth  Annual  Meeting  of  the 
Psychonomic  Society,  Phoenix,  November  1979. 

Collins,  A.M.,  4  Loftus,  E.G.  A  spreading-activation  theory  of  semantic 
processing.  Psychological  Review.  1975,  82,  407-428. 

DeGroot,  A.D.  Perception  and  memory  versus  thought:  Seme  old  Ideas  and 
recent  findings.  In  B.  Klelnmuntz  (Ed.),  Problem  solving;  Research. 
method,  and  theory.  New  York:  Wiley.  196V. 

DeKleer,  J.  Casual  and  teleological  reasoning  in  circuit  recognition. 
(AR-TR-529.)  Cambridge,  MA:  Artificial  Intelligence  Laboratory,  Mass- 
achusetts  Institute  of  Technology,  September  1979. 


45 


DeKleer,  J.,  &  Brown,  J.S.  Mental  models  of  physical  mechanisms  and 
their  acquisition.  Unpublished  manuscript,  1980.  (Available  from  XEROX 
PARC,  Cognitive  and  Institutional  Sciences,  3333  Coyote  Hill  Road,  Palo 
Alto,  CA,  94304.) 

Egan,  D.C.,  &  Schwartz,  B.J.  Chunking  In  recall  of  symbolic  drawings. 
Memory  and  Cognition,  1979,  2»  149-158. 

Greeno,  J.G.  Natures  of  problem-solving  abilities.  In  W.K.  Estes  (Ed.), 
Handbook  of  learning  and  cognitive  processes.  Volume  5:  Human  Informa¬ 
tion  processing.  Hillsdale,  NJ:  Erlbaum,  1978. 

Larkin,  J.H.  Expertise  and  solution  strategies  for  textbook  physics 
problems.  Paper  presented  at  the  Twentieth  Annual  Meeting  of  the 
Psychonomlc  Society,  Phoenix,  November  1979. 

Larkin,  J.H.,  McDermott,  J.,  Simon,  D.P.,  &  Simon,  H.A.  Models  of  compe¬ 
tence  in  solving  physics  problems.  Cognitive  Science,  1980,  4,  317-345. 

Newell,  A.,  &  Simon,  H.A.  Human  problem  solving.  Englewood  Cliffs,  NJ: 
Prentice-Hall,  1972. 

Norman,  D.A.,  &  Rumelhart,  D.E.  (Eds.)  Explorations  In  cognition. 

San  Francisco:  W.H.  Freeman,  1975. 

Reltman,  J.  Skilled  perception  in  Go:  Deducing  memory  structures  from 
Inter-response  times.  Cognitive  Psychology,  1976,  6,  336-356. 

Reltman,  J.S.,  McKelthen,  K.B.,  Reuter,  H.R.,  &  Hlrtle,  S.  Knowledge 
organization  of  expert  and  novice  programmers.  Paper  presented  at  the 
Twentieth  Annual  Meeting  of  the  Psychonomlc  Society,  Phoenix,  November 
1979. 

Reltman,  J.S.,  &  Reuter,  H.H.  Organization  revealed  by  recall  orders 
and  confirmed  by  pauses.  Cognitive  Psychology,  1980,  12,  554-581. 

Sarle,  W.S.  The  ADCLUS  procedure  (SAS  Technical  Report  No.  S-124). 

Cary,  N.C.:  SAS  Institute,  1981. 

Schank,  R.C.  Conceptual  dependency:  A  theory  of  natural  language 
understanding.  Cognitive  Psychology,  1972,  2,  553-631. 

Shannon,  C.E.,  &  Weaver,  W.  The  mathematical  theory  of  communication, 
Urbana,  IL:  University  of  Illinois  Press,  1949. 

Simon,  H.A. ,  &  Chase,  W.G.  Skill  In  chess.  American  Scientist,  1973, 

61,  394-403. 


46 


Simon,  H.A.,  &  Gilmartin,  K.  A  simulation  of  memory  for  chess  positions. 
Cognitive  Psychology,  1973,  5,  29-46. 


VI  i  nograd, 
Psychol ogy 


.  A  program  for  understanding  natural  language.  Cognitive 
1972,  3,  1-191. 


